Computational notebooks, such as Jupyter notebooks, are interactive computing environments that are ubiquitous among data scientists to perform data wrangling and analytic tasks. To measure the performance of AI pair programmers that automatically synthesize programs for those tasks given natural language (NL) intents from users, we build ARCADE, a benchmark of 1082 code generation problems using the pandas data analysis framework in data science notebooks. ARCADE features multiple rounds of NL-to-code problems from the same notebook. It requires a model to understand rich multi-modal contexts, such as existing notebook cells and their execution states as well as previous turns of interaction. To establish a strong baseline on this challenging task, we develop PaChiNCo, a 62B code language model (LM) for Python computational notebooks, which significantly outperforms public code LMs. Finally, we explore few-shot prompting strategies to elicit better code with step-by-step decomposition and NL explanation, showing the potential to improve the diversity and explainability of model predictions.

配备高速数字化器的前端电子设备正在使用并建议将来的核检测器。最近的文献表明，在处理来自核检测器的数字信号时，深度学习模型，尤其是一维卷积神经网络。模拟和实验证明了该领域神经网络的令人满意的准确性和其他好处。但是，仍需要研究特定的硬件加速在线操作。在这项工作中，我们介绍了Pulsedl-II，这是一种专门设计的，专门为事件功能（时间，能量等）从具有深度学习的脉冲中提取的应用。根据先前的版本，PULSEDL-II将RISC CPU纳入系统结构，以更好地功能灵活性和完整性。 SOC中的神经网络加速器采用三级（算术单元，处理元件，神经网络）层次结构，并促进数字设计的参数优化。此外，我们设计了一种量化方案和相关的实现方法（恢复和位移位），以在所选层类型的选定子集中与深度学习框架（例如Tensorflow）完全兼容。通过当前方案，支持神经网络的量化训练，并通过专用脚本自动将网络模型转换为RISC CPU软件，几乎没有准确性损失。我们在现场可编程门阵列（FPGA）上验证pulsedl-ii。最后，通过由直接数字合成（DDS）信号发生器和带有模数转换器（ADC）的FPGA开发板组成的实验设置进行系统验证。拟议的系统实现了60 PS的时间分辨率和0.40％的能量分辨率，在线神经网络推断在信号与噪声比（SNR）为47.4 dB时。

建立具有可信赖性的AI模型非常重要，尤其是在医疗保健等受监管的地区。在解决Covid-19时，以前的工作将卷积神经网络用作骨干建筑，该骨干建筑物易于过度宣告和过度自信做出决策，使它们不那么值得信赖 - 在医学成像背景下的关键缺陷。在这项研究中，我们提出了一种使用视觉变压器的功能学习方法，该方法使用基于注意力的机制，并检查变形金刚作为医学成像的新骨干结构的表示能力。通过对COVID-19胸部X光片进行分类的任务，我们研究了概括能力是否仅从视觉变形金刚的建筑进步中受益。通过使用“信任评分”计算和视觉解释性技术，对模型的可信度进行了定量和定性评估。我们得出的结论是，基于注意力的特征学习方法在建立可信赖的医疗保健深度学习模型方面有希望。

表中的信息可能是文本的重要补充，使基于表的问题答案（QA）具有巨大的价值。处理表的内在复杂性通常会增加模型设计和数据注释的额外负担。在本文中，我们旨在以最少的注释工作开发一个简单的基于表的质量检查模型。由于基于表的质量检查需要问题和表之间的对齐方式以及在多个表元素上执行复杂推理的能力，因此我们提出了一种杂食性的预读方法，该方法既可以消耗自然数据，又提出了合成数据，以使模型具有这些各自的能力。具体而言，鉴于可免费获得的表，我们利用检索将它们与相关的自然句子配对，以进行掩盖预处理，并通过将SQL从表中进行转换为QA损失进行预处理而合成NL问题。我们在几次和完整的设置中都进行了广泛的实验，结果清楚地证明了模型omnitab的优势，最好的多任务方法分别实现了16.2％和2.7％的绝对增益，在128次和完整的设置中也获得了2.7％建立有关Wickitable Questions的最新最新。详细的消融和分析揭示了自然和合成数据的不同特征，从而阐明了杂食性预处理的未来方向。可以在https://github.com/jzbjyb/omnitab上获得代码，预读数据和预算模型。

今天的大部分AI系统都专注于使用自我关注机制和变压器架构在大量多样化的数据中实现令人印象深刻的性能收益。在本文中，我们建议使用外部注意机制增强变压器架构，以带来外部知识和背景。通过将外部信息集成到预测过程中，我们希望减少对更大的模型的需求，并增加AI系统的民主化。我们发现所提出的外部注意机制可以显着提高现有AI系统的性能，使从业者可以轻松地将基础AI模型自定义到许多不同的下游应用程序。特别是，我们专注于勤杂朗语推理的任务，展示所提出的外部注意机制可以增加现有的变压器模型，并显着提高模型的推理能力。拟议的系统，知识外部关注推理（Kear），达到了开放的铜商QA研究基准的人类奇偶校验，其准确性为89.4 \％，与人类准确性为88.9 \％。

The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam. Pursuing the theory behind warmup, we identify a problem of the adaptive learning rate -its variance is problematically large in the early stage, and presume warmup works as a variance reduction technique. We provide both empirical and theoretical evidence to verify our hypothesis. We further propose Rectified Adam (RAdam), a novel variant of Adam, by introducing a term to rectify the variance of the adaptive learning rate. Experimental results on image classification, language modeling, and neural machine translation verify our intuition and demonstrate the efficacy and robustness of RAdam. 1 * Work was done during an internship at Microsoft Dynamics 365 AI. † Work was done during an internship at Microsoft Dynamics 365 AI.

Content-Controllable Summarization generates summaries focused on the given controlling signals. Due to the lack of large-scale training corpora for the task, we propose a plug-and-play module RelAttn to adapt any general summarizers to the content-controllable summarization task. RelAttn first identifies the relevant content in the source documents, and then makes the model attend to the right context by directly steering the attention weight. We further apply an unsupervised online adaptive parameter searching algorithm to determine the degree of control in the zero-shot setting, while such parameters are learned in the few-shot setting. By applying the module to three backbone summarization models, experiments show that our method effectively improves all the summarizers, and outperforms the prefix-based method and a widely used plug-and-play model in both zero- and few-shot settings. Tellingly, more benefit is observed in the scenarios when more control is needed.

Keyword spotting (KWS) based on deep neural networks (DNNs) has achieved massive success in voice control scenarios. However, training of such DNN-based KWS systems often requires significant data and hardware resources. Manufacturers often entrust this process to a third-party platform. This makes the training process uncontrollable, where attackers can implant backdoors in the model by manipulating third-party training data. An effective backdoor attack can force the model to make specified judgments under certain conditions, i.e., triggers. In this paper, we design a backdoor attack scheme based on Voiceprint Selection and Voice Conversion, abbreviated as VSVC. Experimental results demonstrated that VSVC is feasible to achieve an average attack success rate close to 97% in four victim models when poisoning less than 1% of the training data.

Dialogue summarization has recently garnered significant attention due to its wide range of applications. However, existing methods for summarizing dialogues are suboptimal because they do not take into account the inherent structure of dialogue and rely heavily on labeled data, which can lead to poor performance in new domains. In this work, we propose DIONYSUS (dynamic input optimization in pre-training for dialogue summarization), a pre-trained encoder-decoder model for summarizing dialogues in any new domain. To pre-train DIONYSUS, we create two pseudo summaries for each dialogue example: one is produced by a fine-tuned summarization model, and the other is a collection of dialogue turns that convey important information. We then choose one of these pseudo summaries based on the difference in information distribution across different types of dialogues. This selected pseudo summary serves as the objective for pre-training DIONYSUS using a self-supervised approach on a large dialogue corpus. Our experiments show that DIONYSUS outperforms existing methods on six datasets, as demonstrated by its ROUGE scores in zero-shot and few-shot settings.

The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.